<!DOCTYPE HTML PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Deferreds are beautiful! (A Tutorial)</title>
</head>

<body>
<h1>Deferreds are beautiful! (A Tutorial)</h1>

<h2>Introduction</h2>

<p>Deferreds are quite possibly the single most confusing topic that a
newcomer to Twisted has to deal with. I am going to forgo the normal talk
about what deferreds are, what they aren't, and why they're used in Twisted.
Instead, I'm going show you the logic behind what they
<strong>do</strong>.</p>


<p>A deferred allows you to encapsulate the logic that you'd normally use to
make a series of function calls after receiving a result into a single object.
In the examples that follow, I'll first show you what's going to go on behind 
the scenes in the deferred chain, then show you the deferred API calls that set
up that chain. All of these examples are runnable code, so feel free to play 
around with them.</p>

 
<h2>A simple example</h2>

First, a simple example so that we have something to talk about:

<a href="listings/deferred/deferred_ex.py" class="py-listing">deferred_ex.py</a>

<p>And the output: (since both methods in the example produce the same output,
it will only be shown once.) </p>

<pre>
callback 1
        got result: success
</pre>

<p>Here we have the simplest case. A deferred with a single callback and a
single errback. Normally, a function would create a deferred and hand it back
to you when you request an operation that needs to wait for an event for
completion.  The object you called then does <code>d.callback(result)</code>
when the results are in.  
</p>

<p>The thing to notice is that there is only one result that is passed from
method to method, and that the result returned from a method is the argument
to the next method in the chain. In case of an exception, result is set to an
instance of <code class="API" base="twisted.python.failure">Failure</code>
that describes the exception.</p>

<h2>Errbacks</h2>
<h3>Failure in requested operation</h3>
<p>Things don't always go as planned, and sometimes the function that
returned the deferred needs to alert the callback chain that an error
has occurred.</p>

<a href="listings/deferred/deferred_ex1a.py" class="py-listing">deferred_ex1a.py</a>

<pre>
errback
we got an exception: Traceback (most recent call last):
--- exception caught here ---
  File "deferred_ex1a.py", line 73, in ?
    raise RuntimeError, "*doh*! failure!"
exceptions.RuntimeError: *doh*! failure!
</pre>

<p> The important thing to note (as it will come up again in later examples)
is that the callback isn't touched, the failure goes right to the errback.
Also note that the errback trap()s the expected exception type. If you don't
trap the exception, an error will be logged when the deferred is
garbage-collected. 
</p>


<h3>Exceptions raised in callbacks</h3>

<p>Now let's see what happens when <em>our callback</em> raises an
exception</p>

<a href="listings/deferred/deferred_ex1b.py" class="py-listing">deferred_ex1b.py</a>

<p>And the output: (note, tracebacks will be edited slightly to conserve
space)</p>

<pre>
callback 1
        got result: success
callback 2
        got result: yay! handleResult was successful!
        about to raise exception
errback
we got an exception: Traceback (most recent call last):
--- &lt;exception caught here&gt; ---
  File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line
326, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "./deferred_ex1.py", line 32, in failAtHandlingResult
    raise RuntimeError, "whoops! we encountered an error"
exceptions.RuntimeError: whoops! we encountered an error
</pre>

<p>If your callback raises an exception, the next method to be called will be 
the next errback in your chain.</p>


<h3>Exceptions will only be handled by errbacks</h3>

<p>If a callback raises an exception the next method to be called will be next
errback in the chain. If the chain is started off with a failure, the first
method to be called will be the first errback.</p>

<a href="listings/deferred/deferred_ex2.py" class="py-listing">deferred_ex2.py</a>

<pre>
callback 1
        got result: success
        about to raise exception
errback
we got an exception: Traceback (most recent call last):
  File "./deferred_ex2.py", line 85, in ?
    nonDeferredExample("success")
--- &lt;exception caught here&gt; ---
  File "./deferred_ex2.py", line 46, in nonDeferredExample
    result = failAtHandlingResult(result)
  File "./deferred_ex2.py", line 35, in failAtHandlingResult
    raise RuntimeError, "whoops! we encountered an error"
exceptions.RuntimeError: whoops! we encountered an error
</pre>

<p>You can see that our second callback, handleResult was not called because
failAtHandlingResult raised an exception</p>

<h3>Handling an exception and continuing on</h3>

<p>In this example, we see an errback handle an exception raised in the
preceeding callback.  Take note that it could just as easily been an exception
from <strong>any other</strong> preceeding method. You'll see that after the
exception is handled in the errback (i.e.  the errback does not return a
failure or raise an exception) the chain continues on with the next
callback.</p>

<a href="listings/deferred/deferred_ex3.py" class="py-listing">deferred_ex3.py</a>

<pre>
callback 1
        got result: success
        about to raise exception
errback
we got an exception: Traceback (most recent call last):
--- &lt;exception caught here&gt; ---
  File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line
326, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "./deferred_ex2.py", line 35, in failAtHandlingResult
    raise RuntimeError, "whoops! we encountered an error"
exceptions.RuntimeError: whoops! we encountered an error
</pre>

<h2>addBoth: the deferred version of <em>finally</em></h2>

<p>Now we see how deferreds do <strong>finally</strong>, with .addBoth. The
callback that gets added as addBoth will be called if the result is a failure
or non-failure. We'll also see in this example, that our doThisNoMatterWhat()
method follows a common idiom in deferred callbacks by acting as a passthru,
returning the value that it received to allow processing the chain to
continue, but appearing transparent in terms of the result.</p>

<a href="listings/deferred/deferred_ex4.py" class="py-listing">deferred_ex4.py</a>

<pre>
callback 1
        got result: success
callback 2
        got result: yay! handleResult was successful!
        about to raise exception
both 3
        got argument &lt;twisted.python.failure.Failure exceptions.RuntimeError&gt;
        doing something very important
errback
we got an exception: Traceback (most recent call last):
--- &lt;exception caught here&gt; ---
  File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line
326, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "./deferred_ex4.py", line 32, in failAtHandlingResult
    raise RuntimeError, "whoops! we encountered an error"
exceptions.RuntimeError: whoops! we encountered an error
</pre>

<p>You can see that the errback is called, (and consequently, the failure is
trapped).  This is because doThisNoMatterWhat method returned the value it
received, a failure.</p>

<h2>addCallbacks: decision making based on previous success or failure</h2>

<p>As we've been seeing in the examples, the callback is a pair of
callback/errback.  Using addCallback or addErrback is actually a special case
where one of the pair is a pass statement. If you want to make a decision
based on whether or not the previous result in the chain was a failure or not
(which is very rare, but included here for completeness), you use
addCallbacks. Note that this is <strong>not</strong> the same thing as an
addCallback followed by an addErrback.</p>


<a href="listings/deferred/deferred_ex5.py" class="py-listing">deferred_ex5.py</a>

<pre>
callback 1
        got result: success
        about to raise exception
no decision 2
        *doh*! a failure! quick! damage control!
callback 3
        got result: damage control successful!
yes decision 4
        wasn't a failure, so we can plow ahead
callback 5
        got result: go ahead!
</pre>

<p>Notice that our errback is never called. The noDecision method returns a
non-failure so processing continues with the next callback. If we wanted to
skip the callback at &quot;- A -&quot; because of the error, but do some kind of
processing in response to the error, we would have used a passthru, and
returned the failure we received, as we see in this next example: </p>

<a href="listings/deferred/deferred_ex6.py" class="py-listing">deferred_ex6.py</a>

<pre>
callback 1
        got result: success
        about to raise exception
no decision 2
        *doh*! a failure! don't know what to do, returning failure!
no decision 3
        *doh*! a failure! quick! damage control!
callback 4
        got result: damage control successful!
</pre>

<p>Two things to note here. First, &quot;- A -&quot; was skipped, like we wanted it to,
and the second thing is that after &quot;- A -&quot;, noDecision is called, because
<strong>it is the next errback that exists in the chain</strong>. It returns a
non-failure, so processing continues with the next callback at &quot;- B -&quot;, and
the errback at the end of the chain is never called </p>

<h2>Hints, tips, common mistakes, and miscellaney</h2>

<h3>The deferred callback chain is stateful</h3>

<p>A deferred that has been called back will call it's addCallback and
addErrback methods as appropriate in the order they are added, when they are
added. So we see in the following example, deferredExample1 and
deferredExample2 are equivalent. The first sets up the processing chain
beforehand and then executes it, the other executes the chain as it is being
constructed.  This is because deferreds are <em>stateful</em>.  </p>

<a href="listings/deferred/deferred_ex7.py" class="py-listing">deferred_ex7.py</a>

<pre>
callback 1
        got result: success
        about to raise exception
errback
we got an exception: Traceback (most recent call last):
--- &lt;exception caught here&gt; ---
  File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line
326, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "./deferred_ex7.py", line 35, in failAtHandlingResult
    raise RuntimeError, "whoops! we encountered an error"
exceptions.RuntimeError: whoops! we encountered an error


-------------------------------------------------

callback 1
        got result: success
        about to raise exception
errback
we got an exception: Traceback (most recent call last):
--- &lt;exception caught here&gt; ---
  File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line
326, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "./deferred_ex7.py", line 35, in failAtHandlingResult
    raise RuntimeError, "whoops! we encountered an error"
exceptions.RuntimeError: whoops! we encountered an error
</pre>

<p>This example also shows you the common idiom of chaining calls to
addCallback and addErrback.
</p>

<h3>Don't call .callback() on deferreds you didn't create!</h3>

<p>It is an error to reinvoke deferreds callback or errback method, therefore
if you didn't create a deferred, <strong>do not under any
circumstances</strong> call its callback or errback.  doing so will raise
an exception </p>

<h3>Callbacks can return deferreds</h3>

<p>If you need to call a method that returns a deferred within your callback
chain, just return that deferred, and the result of the secondary deferred's
processing chain will become the result that gets passed to the next callback
of the primary deferreds processing chain </p>

<a href="listings/deferred/deferred_ex8.py" class="py-listing">deferred_ex8.py</a>

<pre>
callback 1
        got result: I hope you'll agree: 
sub-callback a
        adding ' are ' to result
sub-callback b
        adding ' beautiful!' to result
callback 2
        got result: I hope you'll agree:  Deferreds  are  beautiful!
</pre>

<h2>Conclusion</h2>

<p>Deferreds can be confusing, but only because they're so elegant and simple.
There is a lot of logical power that can expressed with a deferred's
processing chain, and once you see what's going on behind the curtain, it's a
lot easier to understand how to make use of what deferreds have to offer.</p>

</body> </html>
